A Robust Morphological Approach for Semantic Segmentation of Very High Resolution Images

State-of-the-art methods for semantic segmentation of images involve computationally intensive neural network architectures. Most of these methods are not adaptable to high-resolution image segmentation due to memory and other computational issues. Typical approaches in literature involve design of neural network architectures that can fuse global information from low-resolution images and local information from the high-resolution counterparts. However, architectures designed for processing high resolution images are unnecessarily complex and involve a lot of hyper parameters that can be difficult to tune. Also, most of these architectures require ground truth annotations of the high resolution images to train, which can be hard to obtain. In this article, we develop a robust pipeline based on mathematical morphological (MM) operators that can seamlessly extend any existing semantic segmentation algorithm to high resolution images. Our method does not require the ground truth annotations of the high resolution images. It is based on efficiently utilizing information from the low-resolution counterparts, and gradient information on the high-resolution images. We obtain high quality seeds from the inferred labels on low-resolution images using traditional morphological operators and propagate seed labels using a random walker to refine the semantic labels at the boundaries. We show that the semantic segmentation results obtained by our method beat the existing state-of-the-art algorithms on high-resolution images. We empirically prove the robustness of our approach to the hyper parameters used in our pipeline. Further, we characterize some necessary conditions under which our pipeline is applicable and provide an in-depth analysis of the proposed approach.Comment: Under review at IEEE Transactions on Image Processin

Iterated Watersheds, A Connected Variation of K-Means for Clustering GIS Data

International audienceIn digital age new approaches for effective and efficient governance strategies can be established by exploiting the vast computing and data resources at our disposal. In several cases, the problem of efficient governance translates to finding a solution to an optimization problem. A typical example is where several cases are framed in terms of clustering problem-Given a set of data objects, partition them into clusters such that elements belonging to the same cluster are similar and elements belonging to different clusters are dissimilar. For example, problems such as zonation, river linking, facility allocation and visualizing spatial data can all be framed as clustering problems. However, all these problems come with an additional constraint that the clusters must be connected. In this article, we propose a suitable solution to the clustering problem with a constraint that the clusters must be connected. This is achieved by suitably modifying K-Means algorithm to include connectivity constraints. The modified algorithm involves repeated application of watershed transform, and hence is referred to as iterated watersheds. This algorithm is analyzed in detail using toy examples and the domain of image segmentation due to wide availability of labelled datasets. It has been shown that iterated watersheds perform better than methods such as spectral clustering, isoperimetric partitioning, and K-Means on various measures. To illustrate the applicability of iterated watersheds-a simple problem of placing emergency stations and suitable cost function is considered. Using real world road networks of various cities, iterated watersheds is compared with K-Means and greedy K-center methods. It has been shown that iterated watersheds result in very good improvements over these methods across various experiments

Neither Cholinergic Nor Dopaminergic Enhancement Improve Spatial Working Memory Precision in Humans

Acetylcholine and dopamine are neurotransmitters that play multiple important roles in perception and cognition. Pharmacological cholinergic enhancement reduces excitatory receptive field size of neurons in marmoset primary visual cortex and sharpens the spatial tuning of visual perception and visual cortical fMRI responses in humans. Moreover, previous studies show that manipulation of cholinergic or dopaminergic signaling alters the spatial tuning of macaque prefrontal cortical neurons during the delay period of a spatial working memory (SWM) task and can improve SWM performance in macaque monkeys and human subjects. Here, we investigated the effects of systemic cholinergic and dopaminergic enhancement on the precision of SWM, as measured behaviorally in human subjects. Cholinergic transmission was increased by oral administration of 5 mg of the cholinesterase inhibitor donepezil, and dopaminergic signaling was enhanced with 100 mg levodopa/10 mg carbidopa. Each neurotransmitter system was separately investigated in double-blind placebo-controlled studies. On each trial of the SWM task, a square was presented for 150 ms at a random location along an invisible circle with a radius of 12 degrees of visual angle, followed by a 900 ms delay period with no stimulus shown on the screen. Then, the square was presented at new location, displaced in either a clockwise (CW) or counterclockwise (CCW) direction along the circle. Subjects used their memory of the location of the original square to report the direction of displacement. SWM precision was defined as the amount of displacement corresponding to 75% correct performance. We observed no significant effect on SWM precision for either donepezil or levodopa/carbidopa. There was also no significant effect on performance on the SWM task (percent correct across all trials) for either donepezil or levodopa/carbidopa. Thus, despite evidence that acetylcholine and dopamine regulate spatial tuning of individual neurons and can improve performance of SWM tasks, pharmacological enhancement of signaling of these neurotransmitters does not substantially affect a behavioral measure of the precision of SWM in humans

Power Tree Filter: A Theoretical Framework Linking Shortest Path Filters and Minimum Spanning Tree Filters

International audienceEdge-preserving image filtering is an important pre-processing step in many filtering applications. In this article, we analyse the basis of edge-preserving filters and also provide theoretical links between the MST filter, which is a recent state-of-art edge-preserving filter, and filters based on geodesics. We define shortest path filters, which are closely related to adaptive kernel based filters, and show that MST filter is an approximation to the Γ −limit of the shortest path filters. We also propose a different approximation for the Γ −limit that is based on union of all MSTs and show that it yields better results than that of MST approximation by reducing the leaks across object boundaries. We demonstrate the effectiveness of the proposed filter in edge-preserving smoothing by comparing it with the tree filter

An Introduction to Gamma-Convergence for Spectral Clustering

International audienceThe problem of clustering is to partition the dataset into groups such that elements belonging to the same group are similar and elements belonging to the different groups are dissimilar. The unsupervised nature of the problem makes it widely applicable and also tough to solve objectively. Clustering in the context of image data is referred to as image segmentation. Distance based methods such as K-means fail to detect the non-globular clusters and hence spectral clustering was proposed to overcome this problem. This method detects the non glob-ular structures by projecting the data set into a subspace, in which the usual clustering methods work well. Gamma convergence is the study of asymptotic behavior of minimizers of a family of minimization problems. Such a limit of minimizers is referred to as the gamma limit. Calculating the gamma limit for various variational problems have been proved useful giving a different algorithm and insights into why existing methods work. In this article, we calculate the gamma limit of the spectral clustering methods, analyze its properties, and compare them with minimum spanning tree based clustering methods and spectral clustering methods

Triplet-Watershed for Hyperspectral Image Classification

International audienceHyperspectral images (HSI) consist of rich spatial and spectral information, which can potentially be used for several applications. However, noise, band correlations and high dimensionality restrict the applicability of such data. This is recently addressed using creative deep learning network architectures such as ResNet, SSRN, and A2S2K. However, the last layer, i.e. the classification layer, remains unchanged and is taken to be the softmax classifier. In this article, we propose to use a watershed classifier. Watershed classifier extends the watershed operator from Mathematical Morphology for classification. In its vanilla form, the watershed classifier does not have any trainable parameters. In this article, we propose a novel approach to train deep learning networks to obtain representations suitable for the watershed classifier. The watershed classifier exploits the connectivity patterns, a characteristic of HSI datasets, for better inference. We show that exploiting such characteristics allows the Triplet-Watershed to achieve state-of-art results in supervised and semi-supervised contexts. These results are validated on Indianpines (IP), University of Pavia (UP), Kennedy Space Center (KSC) and University of Houston (UH) datasets, relying on simple convnet architecture using a quarter of parameters compared to previous state-of-the-art networks.The source code for reproducing the experiments and supplementary material (high resolution images) is available at https://github.com/ac20/TripletWatershed_Code

Revisiting the Isoperimetric Graph Partitioning Problem

International audienceIsoperimetric graph partitioning which is also known the Cheeger cut is NP-Hard in its original form. In literature, multiple modifications to this problem have been proposed to obtain approximation algorithms for clustering applications. In the context of image segmentation, a heuristic continuous relaxation to this problem has yielded good quality results. This algorithm is based on solving a linear system of equations involving the Laplacian of the image graph. Further, the same algorithm applied to a maximum spanning tree (MST) of the image graph was shown to produce similar results at a much lesser computational cost. However, the data reduction step (i.e. considering a MST, a much sparser graph compared to the original graph) leading to a faster yet useful algorithm has not been analysed. In this article, we revisit the isoperimetric graph partitioning problem and rectify a few discrepancies in the simplifications of the heuristic continuous relaxation, leading to a better interpretation of what is really done by this algorithm. We then use the Power Watershed (PW) framework to show that is enough to solve the relaxed isoperimetric graph partitioning problem on the graph induced by Union of Maximum Spanning Trees (UMST) with a seed constraint. The UMST has a lesser number of edges compared to the original graph, thus improving the speed of sparse matrix multiplication. Further, given the interest of PW framework in solving the relaxed seeded isoperimetric partitioning problem, we discuss the links between the PW limit of the discrete isoperimetric graph partitioning and watershed cuts. We then illustrate with experiments, a detailed comparison of solutions to the relaxed seeded isoperimetric partitioning problem on the original graph with the ones on the UMST and a MST. Our study opens many research directions which are discussed in the conclusions section

Some Theoretical Links Between Shortest Path Filters and Minimum Spanning Tree Filters

International audienceEdge-aware filtering is an important pre-processing step in many computer vision applications. In literature, there exist several versions of collaborative edge-aware filters based on spanning trees and shortest path heuristics which work well in practice. For instance, Tree Filter (TF) which is recently proposed based on a minimum spanning tree (MST) heuristic yields promising results in many filtering applications. However, links between the tree-based filters and shortest path-based filters are faintly explored. In this article, we introduce an edge-aware generalization of the TF, termed as UMST filter based on the union of all MSTs. The major contribution of this paper is establishing theoretical links between filters based on MSTs and filters based on geodesics via power watershed framework. More precisely, we show that Union of Minimum Spanning Trees (UMST) filter can be obtained as the limit of Shortest Path Filters (SPFs). Intuitively, TF can be viewed as an approximate limit of the SPFs. We propose and provide a detailed analysis of two different implementations of the UMST filter based on shortest paths. Further, we establish empirically with the help of denoising experiments that TF is an approximate limit by showing that TF and one of our approximations yield similar results

Watersheds for Semi-Supervised Classification

International audienceWatershed technique from mathematical morphology (MM) is one of the most widely used operators for image segmentation. Recently watersheds are adapted to edge weighted graphs, allowing for wider applicability. However, a few questions remain to be answered-(a) How do the boundaries of the watershed operator behave? (b) Which loss function does the watershed operator optimize? (c) How does watershed operator relate with existing ideas from machine learning. In this article, a framework is developed, which allows one to answer these questions. This is achieved by generalizing the maximum margin principle to maximum margin partition and proposing a generic solution, MORPHMEDIAN, resulting in the maximum margin principle. It is then shown that watersheds form a particular class of MORPHMEDIAN classifiers. Using the ensemble technique, watersheds are also extended to ensemble watersheds. These techniques are compared with relevant methods from literature and it is shown that watersheds perform better than SVM on some datasets, and ensemble watersheds usually outperform random forest classifiers